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Survey Tools for State Education Leaders 


INTRODUCTION. USING FEEDBACK LOOPS TO MONITOR IMPLEMENTATION 


As state education agencies (SEAs) across the nation take on the ambitious work of 
implementing the Common Core State Standards (CCSS), it is critical that they have a clear 
understanding of the effectiveness of their efforts: Are educators being reached with the correct 
messages? Are the SEAs’ supports and resources meeting the needs of the field? What 
challenges exist and require the SEA’s attention? 


To better understand these issues, SEAs can use feedback loops to know whether 
implementation is taking place as intended. Feedback loops allow SEA leaders to know if their 
work is on-track for success, or if challenges exist that require more attention. 


A feedback loop requires two things: 


An objective: what are you trying to learn about the quality of implementation? 
A source of evidence: how will you measure it? Where will you get the evidence? 


There are seven critical objectives for feedback loops on CCSS implementation. The table 
below lists these objectives and some potential sources of evidence for each. 


Figure 1: Feedback loops for CCSS implementation in the classroom 


Objective Potential sources of evidence 
Surveys of educators/school leaders 
Assess awareness and support of the Focus groups/interviews with educators 
CCSS 


Surveys of educators/school leaders 
Gauge understanding of the CCSS Focus groups/interviews with educators 
Results of “pre” and “post” tests of CCSS 
knowledge before and after professional 
development 


Project management data (e.g., status of 


Assess the reach of and satisfaction critical milestones in implementation plan) 
with CCSS resources (e.g., quality and Website hits (e.g., download rates of provided 
alignment of curricular and instructional resources) 

materials) that have been provided Professional development participation 


Formal evaluations of professional 
development offerings and other resources 
Surveys of educators/school leaders 


Focus groups/interviews with educators 
Identify effective communication and Website hits 
outreach mechanisms for CCSS Surveys of educators/school leaders 
information and resources Focus groups/interviews with educators 
Identify challenges to CCSS Project management data 
implementation and potential solutions Surveys of educators/school leaders 
Focus groups/interviews with educators 
Assess changes in classroom practice Surveys of educators/school leaders 
that result from CCSS implementation Focus groups/interviews with educators 
Site/classroom observations 
Review/audit of information from classroom 
observations 
Assess impact of CCSS implementation Compare any of the above measures with 
on student outcomes formative and summative assessment data 


(e.g., did educators who received resources, 
liked them, and/or used them to change their 
practice achieve better results with their 
students?) 


As the table above shows, one commonly-used source of evidence for feedback loops is the 
survey, which is often utilized to gather information from a targeted sample of individuals. Most 
SEAs use surveys to periodically generate feedback on specific professional development 
opportunities or other resources that have been provided to the field. Typically, these surveys 
are administered sporadically, without a broader vision for how the information received can 
improve policy and practice. With a bit more planning and coordination, however, state leaders 
can take their use of surveys to the next level by using them to monitor implementation in more 
rigorous, deliberate ways. 


This document contains guidance for using the accompanying set of “Common Core Survey 
Tools” for educators and instructional leaders. The purpose of administering surveys using 
these tools — and analyzing the data generated from them — is to support CCSS implementation 
efforts by giving state leaders a voluntary means to gather rapid, real-time feedback on the 
quality of implementation before the first common assessment results are available. This 
feedback will allow leaders to make course corrections in their CCSS implementation efforts by 
giving them insight into the inevitable challenges that will arise before it is too late to do 
something about them. 


These tools are primarily aimed at SEA leaders. However, LEA leaders interested in gathering 
feedback from the field should be able to adapt and use them to gauge the quality of 
implementation at the district level as well. 


COMMON CORE SURVEY TOOLS 


The survey questions in this document aim to accomplish six of the seven objectives listed 
above: 


1. Assess respondents’ awareness and support of the CCSS. 


2. Gauge respondents’ understanding of the CCSS. 


3. Assess the reach of and satisfaction with CCSS resources that have been provided. 


4. Identify effective communication and outreach mechanisms for CCSS information and 
resources. 


5. Identify challenges to CCSS implementation. 


6. Assess changes in classroom practice that result from CCSS implementation. 


WHO SHOULD USE THESE TOOLS? 


These tools are primarily designed for state leaders who are supporting CCSS implementation 
via the provision of professional development, the creation of instructional materials, the 
development of new assessment systems, or other functions. Many of the questions are also 
applicable to leaders at the district and school levels who may wish to obtain feedback on their 
own CCSS implementation efforts. 


WHEN AND HOW SHOULD WE ADMINISTER THESE SURVEYS? 


Leaders should administer these surveys as often as they feel they need the feedback. As a 
general rule, it is a good idea to survey major populations of interest (e.g., teachers, principals 
and other teacher leaders) more than once a year if possible. It may also be helpful to time 
surveys of specific populations around major events in CCSS implementation; for example, the 
survey questions may be used to create “pre” and “post” assessments of educator awareness 
and knowledge before and after summer professional development offerings. 


HOW SHOULD WE USE THESE TOOLS TO DEVELOP A SURVEY? 


Given the complexity of CCSS implementation, it is highly recommended that SEA leaders begin 
by convening a team to lead the process of creating feedback loops, comprised of those 
responsible for key areas of implementation. This need not be an additional meeting or routine; 
in fact, it is likely that there are one or more groups that already convene regularly at the SEA 
to discuss CCSS implementation. If this is the case, feedback loops can become another 
dimension of their work together. 


There are a number of steps that SEA leaders can take to structure a survey to meet their 
specific needs. These steps are generally good practice for preparing to administer any survey: 


Step one: Identify the purpose of the survey and the targeted audience. 


It is critical that the survey team has a shared understanding about why the survey is being 
administered. This clarity will help ensure that everyone is on the same page as they undertake 
this work together. 


Key questions to consider: 


Which of the objectives on page 2 are most aligned to the SEA’s needs? 
Who is being targeted for feedback? 

What information does the team want to collect? 

Who will be the primary users of the data? 


Step two: Take context into account. 


Before diving in to create the survey, the survey team should be clear about the state’s work- 
to-date and future efforts on the horizon. 


Key questions to consider: 


e What is the state’s timeline for implementing the new standards? Are the standards 
being phased in by grade or content area, or both? Are there districts that are ahead of 
the state timeline? If so, should survey administration look different for them? 

e Based on this timeline, when must state leaders make key decisions on how to invest 
time and resources? 

e What are the state’s critical dates for CCSS implementation? For instance, are there any 
large convenings scheduled? Is a model curriculum being released? Is a new resource 
for educators under development? These critical dates may have implications for when 
state leaders should gather feedback from the field. 

e What have been the SEA’s implementation challenges in the past? When working to 
create the survey, state leaders can draw from their past experiences to better 
understand where weaknesses might exist and where more attention might be needed. 
With this knowledge, they can prioritize areas where feedback would be most helpful. 


Step three: Address logistics. 


The devil is in the details, as the saying goes. There’s no doubt: taking the time to think 
through logistics can save time and energy down the line. 


Key questions to consider: 


e How often should the survey be administered to obtain the information needed? Just 
once, or periodically? 

e In what format will the survey be administered: hard copy or electronically? 

e Who will create the survey(s)? For electronic versions this work may be more time 
consuming than paper versions. However, using an online survey tool (like 
SurveyMonkey or Zoomerang) can make data analysis much easier. 

e Who will compile the data? 

e Who will be responsible for getting the survey into the hands of respondents? Your 
survey may present a good opportunity to work with other state partners (teachers’ 
union, principals’ association, regional service centers, etc.) or to reach out to school 


and district leaders for their assistance. Not only can these partnerships make your 
survey administration much more successful, they can also help generate more buy-in 
for your efforts. 


Step four: Create a plan. 


Using the information and insights generated from the three steps above, create a plan to 
administer the survey. The template in Figure 2 below can be used to capture the most 
important elements of your survey administration plans. 


Figure 2: Survey Planning Template 


Survey Planning Template 


Name of Survey: 
Targeted audience: 
Purpose of survey: 
Related events: 


Type of survey (hard copy or electronic): 


Lead SEA staff for key roles 

Drafting the survey: 

Building the survey (for electronic versions): 
Administering survey to respondents: 

Compiling survey data and summarizing findings: 
External partners involved in survey administration: 


Administration timeline 


Survey document approved: 
Survey release date(s): 
Survey close date (s): 


Complete data analysis and findings: 


If your team plans to administer more than one survey over a period of time, consider creating 
a timeline to show which surveys will be administered and when. The example in Figure 3 is 
one approach for how a timeline can be organized. 


Figure 3: Survey administration timeline 
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2012 2013 


Target groups Jun Jul Aug Sept Oct Nov Mar Apr May 


= CCSS Summer Institute 
participants A 


" Teaching and learning 
conference participants 


* Curriculum coordinator 
network 


= Principals (statewide) 


= Pre-K — 5 educators 
(statewide) 


= MS and HS educators 
(statewide) 


A Survey Administration 


Step five: Customize the survey. 


The surveys included here are generic templates meant to be customized by survey teams. 
They are organized by objective to facilitate the process of your team selecting the questions 
that will be most useful. Here are some tips to consider as you read through the templates and 
adapt them for your purposes: 


e Choose only the questions that are most likely to generate the information you need. As 
you prioritize some questions over others, be sensitive to the time needed to complete 
the survey you create. Typically, surveys that take more than 5-10 minutes to complete 
are considered too long. 

e There are a variety of question types included in this survey, all with different 
implications for your data analysis. For instance, open ended questions have the benefit 
of providing respondents the opportunity to provide their personal feedback, but they 
can be time-consuming to analyze when looking at hundreds, or even thousands of 
responses. Conversely, multiple choice responses are very easy to code for data 
analysis, but do not typically provide respondents an opportunity to go in to much depth 
with their feedback. Be sure to choose question types that best match your needs and 
capacity to analyze data. 

e After identifying the questions that would be most helpful to you, be sure to edit them 
as needed. Phrases in brackets within the template indicate places where the language 


needs to be customized or removed prior to being released publicly. Update these areas 
with the correct information. 

e You may opt to ask for respondents to provide the name of their school and/or district 
as part of this survey. If you choose to include these questions, it is highly 
recommended that they be clearly listed as optional, and be placed at the end of the 
survey so individuals are fully aware of what will be shared prior to providing their 
information. 

e Explore ways you can also use this survey as a communication tool. At the beginning 
and end of each survey there is space provided to add information specific to your 
state’s implementation of the CCSS. In these spaces, consider including brief 
background information about the state’s adoption of the CCSS, links to important 
resources like the state department’s CCSS website, or the websites of national 
resources on the CCSS. Some national resources could include: 


> Common Core State Standards Initiative: http://www.corestandards.org/ 


This website is produced with the collaboration of the National Governor’s Association (NGA) 
and the Council of Chief State School Officers (CCSSO)—the two organizations that supported 
the state-led process to create the CCSS. The website provides links to download the standards, 
background information about how the standards were developed, and updated information 
about standards adoption across the nation. 


> Achieve: http://www.achieve.org/achieving-common-core 


Achieve has developed materials to help states, districts, and others understand the 
organization and content of the standards and the content and evidence base used to support 
the standards. This portion of the website includes advocacy and communication tools, 
instructional support and alignment resources, implementation planning tools, and highlights 
from state materials and websites. 


> Partnership for Assessment of Readiness for College and Careers (PARCC): 
http://www.parcconline.org 
> Smarter Balanced Assessment Consortium (SBAC): http://www.smarterbalanced.org 


These websites include information on PARCC and SBAC, two consortia of states working to 
create common sets of K-12 assessments aligned to the CCSS. They provide background 
information about the creation of the assessment, in addition to free resources for educators 
and instructional leaders. 


> Student Achievement Partners: http://achievethecore.org 
This website includes free tools and resources to help teachers implement the” shifts” required 


to incorporate the CCSS into their classroom practice. It also provides a compilation of external 
resources related to CCSS implementation. 


Step six: Test the survey. 


Though these survey questions have been tested with national policy experts, authors of the 
CCSS, and educators, it will still be important for you to test it with educators and instructional 
leaders in your system before releasing it broadly. Ask a small group of educators to take the 
survey and then report back on their experience of it. Were the questions clear? Did they make 
sense, given your system's context? What message did the survey send to educators — and was 
anything unintentional conveyed? This feedback will allow you to finalize the survey for the final 
step in the process. 


Step seven: Send out the survey. 


After you complete drafting the survey, send it out broadly. Be sure to schedule time after the 
survey period closes for the survey team to compile and review the data. 


THE SURVEY RESULTS ARE IN; NOW WHAT? 


After your team has successfully created and administered the survey, it is time to turn the piles 
(or spreadsheets) of raw data into useful, actionable information. Your work now turns to data 
analysis and identifying ways to use the data to improve the SEA’s efforts. 


Analyzing your data 


One basic approach to analyzing your data is to identify overall trends in the responses. One 
way to organize the data is to summarize the number of respondents that answered each 
question and the percentages each answer option received. This data could be presented in 
chart format or with graphs. An example is provided in Figure 4a. 


Figure 4a: Example of basic data analysis 


Responses by ELA teachers: "I believe that the Common Core 
will lead to improved student learning for the majority of 
students I serve." 
3% 
@ Strongly Agree 
m Agree 


m= Disagree 


@ Strongly Disagree 


ml don’t know 


The same information can be presented in multiple formats depending on your audience, as 
Figure 4b shows. 


Figure 4b: Example of basic data analysis with infographic 


Responses by ELA teachers: “I believe that the Common Core will lead 
to improved student learning for the majority of students I serve” 
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Most online survey administration tools include functions that help users create charts and 
graphs to display this kind of information. In addition, Microsoft Excel has easy-to-use wizards 
to help create charts and graphs as well. 


A more in-depth approach to analyzing your data could involve the use of cross-tabulations 
(cross-tabs) to show side-by-side comparisons of two or more survey questions to determine 
how they are interrelated. This approach allows you to quickly compare how different groups of 
respondents answered your survey questions. 


Cross-tabs can be created using any one of the characteristics collected in the survey (such as 
grade level or subject taught, or affiliated school or district), and then cross-referencing it with 
any question on the survey. Cross-tabs can also be used to cross-reference two questions 
against one another. Figure 5 is an example of a cross-tab analysis. 


Figure 5: Example of cross-tab analysis 


Question. I believe that the Common Core will lead to improved student learning for the 
majority of students I serve. 


Please identify your 
Total role/ title 
Responses Teacher School 
Counselor 
Answer 
Options 150 100 50 
Strongly Agree 20 a 20 
gly "9 60% 70% 40% 
30 15 15 
sOtee 20% 15% 30% 
eS 15 10 5 
ee 10% 10% 10% 
Strongly 5 0 5 
Disagree 3.3% 0% 10% 
10 5 5 
I tk 
don’t know 6.794 5% 10% 


A number of online survey administration tools can help users create cross-tabs with their 
survey data. Also, in Microsoft Excel users can create pivot tables to create cross-tab charts. 


Figure 6 provides a few ideas of cross-tabs that may provide interesting data for deeper 
analysis on some important areas of implementation. 
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Figure 6: Sample cross-tabs for CCSS survey 


Focus Question 


Possible Cross-Tab Analysis 


Do respondents feel prepared 
to support CCSS 
implementation? 


Analyze educators’ perceived preparedness to teach the CCSS 
(objective 1/#7) by grade level. 


Do respondents recognize 
CCSS-aligned instructional 
practices as important to 
student learning? 


Analyze respondents’ identification of CCSS-aligned practices 
as “important” or “very important” (objective 2/#3 and #6) by 
educator’s subject area. 


Are respondents satisfied with 
the resources provided by the 
department? 


Analyze respondents’ perceptions of the quality of the 
department’s resources (objective 3/#1) by educator's subject 
area or instructional leader’s title. 


Are respondents being 
supported to teach the CCSS? 


Analyze respondents’ perceptions of their professional 
development experiences (objective 3/#7) by grade level or 
subject area. 


Analyze respondents’ identification of a CCSS point of contact 
(objective 4/#1) by their school or district. 


What will be the challenges to 
CCSS implementation across 
the state? 


Analyze respondents’ perceived challenges to CCSS 
implementation (objective 5/#1) by grade level served, 
affiliated school or district, or instructional leader’s title. 


What changes are respondents 
making in their practice as a 
result of the CCSS? 


Analyze educators’ self-reported changes in practice (objective 
6/#2) by grade level served or affiliated school or district. 


Analyze instructional leaders’ self-reported changes in practice 
(objective 6/#7) by title or affiliated district. 


Using data to spur action 


Once you have compiled the data and identified trends within it, work to translate your findings 
into action. This work should involve convening critical stakeholder groups to review the data 
and delving more deeply into areas needing further investigation. 


Convene critical stakeholder groups to review the data 


Survey data can be a powerful lever to drive planning and collaboration. To maximize the use of 
your data, consider convening stakeholder groups to identify trends within the data, make plans 
to address those trends, and identify areas in need of further research. 


Some key groups you may wish to convene include SEA staff responsible for different areas of 
CCSS implementation (including those involved in curriculum and instruction, communications, 
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assessment, and educating special populations); external stakeholders, such as the teachers’ 
union, principals’ association, and regional service centers; and school and district leaders. 


Use these convenings to explore the following questions: 
« What patterns exist in the data? What areas show promise? What areas show challenge? 
« Do obvious solutions exist? 
« If not, what data will help us identify solutions? 


« What are the implications of this data on our work? What are we doing that seems to 
be most helpful? How can we expand it? Is there anything that is hindering progress 
and needs to be rethought? 


« What areas require further investigation? 


These meetings should be structured to lead to participants toward developing new ways of 
addressing the challenges that exist. For example, if the data show that middle grades teachers 
are the least likely to feel prepared to teach the CCSS, SEA professional development staff could 
seek out ways they can collaborate with professional associations to target this group of 
teachers in particular. Or, if a large percentage of respondents express misunderstandings or 
apprehensions about CCSS implementation, communications staff could modify the SEA’s CCSS 
communications strategy to address those concerns in particular. Regardless of the focus area, 
these meetings should be used as opportunities to generate ideas and set a path toward 
improvement. 


Delve into areas needing further investigation 


In many cases, the information that is produced by surveys opens doors to ask more questions 
and research different avenues of your implementation effort. If so, consider incorporating 
other feedback loops to explore these areas in more depth. 


The table in Figure 1 gives some options for creating additional feedback loops. For example: 


e You may wish to learn more about districts that seem to be showing promising signs of 
CCSS implementation. In this circumstance, a few targeted site visits to these districts 
or interviews with leaders in them may help you to get the information you need. 


e You may wish to dig deeper into the implementation challenges identified by educators 
on the survey. In this case, focus groups with educators in the most challenged districts 
may be a good option to learn more. 


e You may want to learn more directly about whether practices are changing in the 
classroom. If your state is implementing a teacher evaluation system, you may begin 
by “auditing” some of the information that is beginning to come out of that system — for 
example, by convening focus groups of principals about what they are observing, or by 
reading samples of observation records filled out by teacher leaders. You may follow 
up on some of this work by visiting some of the referenced classrooms to see if there is 
evidence of progress. 


In any of these cases, the survey questions should serve as a guide for how to structure your 
inquiry. They can be adapted to form focus group or interview protocols, or can serve as 
guiding questions for your team as they look at other evidence. 
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CONCLUSION 


State leaders have recognized that the successful implementation of the CCSS requires 
innovative approaches in a range of areas. In particular, educators will need to be well- 
supported by clear communication of expectations, high-quality professional development, 
aligned curricular and instructional materials, and formative assessment tools. The far-reaching 
scope of this work requires that states strengthen their efforts to receive feedback from the 
field on an ongoing basis. Feedback loops can fill this need by helping state leaders feel 
confident that their efforts are likely to pay off and result in increased student achievement. 
The alternative to this approach—waiting until 2015 to know whether implementation was 
successful—is simply not acceptable. The stakes are far too high for our schools and students. 
Feedback loops can play a critical role in SEAs’ implementation efforts, and can help ensure that 
today’s efforts translate into long-term, sustainable improvements in student learning in the 
years to come. 
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